The development of an instrument for assessing information security in organizations: Examining the content validity using quantitative methods

Content validity, the extent to which a measurement reflects the specific intended domain of content, is a basic type of validity for a valid measurement. It has usually been examined using qualitative methods and has not been given as much attention as the other psychometric properties such as internal consistency reliability, indicator reliability and construct validity in the IS field. In this paper, a quantitative approach including the proportion of substantive agreement (PSA), and substantive validity (CSV) was used to examine content validity for 80 items covering eighth domains related to organizational and individual perspectives of information security. The content validity for the organizational perspective was examined using data from a total of 56 content domain experts. Data from 51 experts were further used to examine content validity for the individual perspective of information security. 31 items did not have an adequate content validity, leaving the instrument with 49 items that have been evaluated for their content validity and can be used in future empirically tests of hypotheses in the information security field. To the knowledge of the authors this quantitative method to assess content validity of items in the process of developing instruments hasn’t yet been applied in the field information security.